Purpose and scope
This is an R Markdown document illustrating my stability analysis of the SARS-Cov2 S1 IgG antibody ELISA data. This data was generated by Ana, where 45 samples (1 full plate and a partial plate of 5 samples) were processed and read in 5 technical replicates each, each of which had 405nm absorbance readings taken at 15, 30, and 45 minutes.
The purpose of this document is to explore, visualize, and summarize the intra and inter-assay variability of these readings.
Visualizing the reaction development
To begin with, I visualize how the absorbance values develop over time for each plate (P_1032 and P_12345), for each replicate of the plate (A, B, C, D, and E). Each individual unknown sample is run in duplicate on each plate. The wells on the first half of the plate are drawn with solid lines, the wells on the second half of the plate are drawn with dotted lines.
There are a few important things to notice here:
- The blank and negative control samples are consistently low but increase slightly from 15 to 45 minutes
- The positive control samples are consistently high
- Most unknown samples in this batch have absorbances between the negative and positive controls
- The absorbances strictly increase from 15 to 30 to 45 minutes
- The sharpest increase in absorbance is from 15 to 30 minutes
Visualizing intra-assay variation
To get an idea of the intra-assay variation, I calculate three metrics from the absorbances (abs) of each unknown and negative control sample, \(i\), and replicates 1 and 2:
the range, which is simply the difference between the largest and smallest value \[\textrm{range}_i = |abs_1 - abs_2|_i\]
the percentage difference, which is the range expressed as a percentage of the mean \[\textrm{percentage difference}_i = \frac{|abs_1 - abs_2|_i} {\bigg(\frac{abs_1 + abs_2}{2}\bigg)_i} \times 100\]
the percentage coefficient of variation, which is the standard deviation expressed as a percentage of the mean \[\textrm{% CV}_i = \frac{\textrm{standard deviation}_i}{\textrm{mean}_i} \times 100\]
For the positive controls and blanks, the same metrics are calculated, but using all the wells of the same type for that plate, at that time point.
These three metrics are plotted as line plots below. There are a few important observations:
- variation does not seem to change much with time, only samples with very high values, such as the positive controls, seem to decrease in variability slightly with time (though this is probably because they are saturating)
- the three metrics show different things and need to be interpreted differently
- the high percentage difference and CVs of the blanks are due to their small absorbance values; small absolute deviations result in a large percentage difference
- there is more variation in absoarbance values between replicates of a plate, than there is between time points on the same plate
- the ranges and CVs are quite high
Absorbance range
Percentage difference
%CV
Next, I plot them as heatmaps where the labels underneath show the combination of plate ID and plate replicate (A to E). The x and y labels correspond to the plate coordinates. Note that the value for the blanks is shown in A6, the value for the negative control is shown in F6 and the value for the positive controls in G6. These plots just help us spatially refer the values to the position on the plates.
There are a few important observations:
- for any particular plate, as you look from 15 minutes (top row) to 45 minutes (bottom row), there is a pattern where the intra-assay variation for a particular well propogates through time, i.e. if a sample has a large discrepancy between its two replicates at 15 minutes, that discrepancy will likely be there at 45 minutes
- samples that start with a large discrepancy between their replicates tend to have smaller ranges and CVs at later time points
- blanks tend to have small ranges but large percentage differences and CVs
- in some plate replicates, the negative and positive controls have unexplained high CVs/ranges
Absorbance range
Percentage difference
%CV
Next, I plot each well as a point. These plots help us to compare the variation between the sample types and between each plate replicate. There are some important observations:
- the positive controls are typically the most reproducible within each plate, followed by the negative control
- there is quite a strong relationship between the control variation and the unknown variation: if the CV of the controls are high, the CV of the unknowns are also high
- there is a large range of variation of the unknowns
Absorbance range
Percentage difference
% CV
Calculating intra-assay variation for each sample type
To summarize the intra-assay variation, I calculate the mean range, percentage difference, and % CV for each sample type, at each time point. The means ± 1 standard deviation are shown.
Note that for the positive control and the unknown samples, all three metrics decrease from 15 minutes to 30 minutes to 45 minutes. This data does not contain the information needed however, to conclude if this increase in precision comes at the cost of reduced assay sensitivity (the ability to discriminate positive cases).
| Type | Time | Range | Percent_diff | CV |
|---|---|---|---|---|
| Blank | 15 | 0.04 ± 0.02 | 48.43 ± 17.99 | 17.11 ± 6.32 |
| Blank | 30 | 0.09 ± 0.07 | 74.27 ± 31.6 | 29.76 ± 15.89 |
| Blank | 45 | 0.15 ± 0.13 | 89.75 ± 38.75 | 38.79 ± 22.07 |
| Negative | 15 | 0.01 ± 0.01 | 13.48 ± 9.19 | 9.53 ± 6.5 |
| Negative | 30 | 0.02 ± 0.02 | 12.55 ± 13.2 | 8.87 ± 9.34 |
| Negative | 45 | 0.03 ± 0.04 | 13.15 ± 15.02 | 9.3 ± 10.62 |
| Positive | 15 | 0.26 ± 0.17 | 11.69 ± 8.04 | 5.23 ± 3.76 |
| Positive | 30 | 0.21 ± 0.15 | 6.61 ± 4.91 | 2.85 ± 2.08 |
| Positive | 45 | 0.1 ± 0.05 | 3.04 ± 1.62 | 1.34 ± 0.7 |
| Unknown | 15 | 0.17 ± 0.14 | 11.99 ± 10.21 | 8.48 ± 7.22 |
| Unknown | 30 | 0.16 ± 0.16 | 8.36 ± 9.26 | 5.91 ± 6.55 |
| Unknown | 45 | 0.14 ± 0.16 | 6.24 ± 8.22 | 4.41 ± 5.81 |
Visualizing inter-assay variation
In addition to evaluating the variation in absorbances within the same plate, it's important to evaluate the variation in absorbances of the same sample across different plates. Here I repeat a similar methodology as for the intra-assay analysis.
To get an idea of the inter-assay variation, I calculate three metrics from the absorbances (abs) of each unknown and negative control sample, \(i\), in the same well across each plate replicate, \(j\). Here, \(n\) is the number of plate replicates.
the range, which is simply the difference between the largest and smallest value \[\textrm{range}_{ij} = max(abs_{i \in j}) - min1(abs_{i \in j})\]
the percentage difference, which is the range expressed as a percentage of the mean \[\textrm{percentage difference}_{ij} = \frac{max(abs_{i \in j}) - min(abs_{i \in j})} {\bigg(\frac{1}{n} \sum_{j=1}^{n} abs_{ij}\bigg)} \times 100\]
the percentage coefficient of variation across the plates, which is the standard deviation expressed as a percentage of the mean \[\textrm{% CV}_{ij} = \frac{\textrm{standard deviation}_{ij}}{\textrm{mean}_{ij}} \times 100\]
This time, I'll start with the heatmaps. The interpretation is the same as before, except now we are looking at the variability of the absorbances of each well across multiple plate replicates.
There are a few important observations: - plate range and percentage difference increase with time, but % CV slightly decreases - discrepancies between plates at 15 minutes generally propogate through to 30 and 45 minutes - the range and percentage difference of absorbances between plates are high for the positive controls, while the CV is low. The opposite is true for the blanks and negative controls - there are no common spatial patterns as to which wells have the largest discrepancies between runs
Absorbance range
Percentage difference
% CV
Next, I plot these as points. This helps compare the sample type stability across plates. There are a few important observations: - the positive controls and many of the unknowns have a very large between-plate range of absorbance values. The opposite is true for the blanks and negative controls - Any change in stability with time is subtle or non-real - On plate P_1032, there are two clusters of unknown samples with different between-plate ranges, though this is not apparent for the other two metrics - the between-plate percentage difference and CV are relatively constant across sample types
Absorbance range
Percentage difference
% CV
Calculating inter-assay variation for each sample type
To summarize the inter-assay variation, I calculate the mean range, percentage difference, and % CV for each sample type, at each time point. The means ± 1 standard deviation are shown.
Note that, just like with the intra-assay variability, for the positive control and the unknown samples, all three metrics decrease from 15 minutes to 30 minutes. This data does not contain the information needed however, to conclude if this increase in precision comes at the cost of reduced assay sensitivity (the ability to discriminate positive cases).
| Type | Time | Range | Percent_diff | CV |
|---|---|---|---|---|
| Blank | 15 | 0.08 ± 0.03 | 100 ± 29.06 | 25.03 ± 7.35 |
| Blank | 30 | 0.11 ± 0.06 | 100 ± 42.56 | 39.65 ± 15.77 |
| Blank | 45 | 0.14 ± 0.1 | 100 ± 51.88 | 51.3 ± 20.24 |
| Negative | 15 | 0.11 ± 0.02 | 100 ± 17 | 11.71 ± 5.51 |
| Negative | 30 | 0.15 ± 0.04 | 100 ± 21.03 | 14.9 ± 7.39 |
| Negative | 45 | 0.2 ± 0.05 | 100 ± 23.14 | 17 ± 7.69 |
| Positive | 15 | 2.31 ± 0.44 | 100 ± 18.94 | 14.83 ± 1.17 |
| Positive | 30 | 3.14 ± 0.27 | 100 ± 8.53 | 6.69 ± 1.09 |
| Positive | 45 | 3.36 ± 0.11 | 100 ± 3.29 | 2.57 ± 0.59 |
| Unknown | 15 | 1.62 ± 0.74 | 100 ± 31.31 | 26.01 ± 7.7 |
| Unknown | 30 | 2.43 ± 0.83 | 100 ± 24.02 | 19.19 ± 8.83 |
| Unknown | 45 | 2.82 ± 0.73 | 100 ± 18.65 | 13.46 ± 9.27 |
Conclusions
Here are a summary of my thoughts based on this analysis:
- the unnormalised and normalised measures of precision are higher than would be expected, but are perhaps not as high as initially thought
- the estimated intra-assay CVs for serum samples are 8.48, 5.91, and 4.41 for 15, 30, and 45 minutes, respectively
- the estimated inter-assay CVs for serum samples are 26.01, 19.19, 13.46 for 15, 30, and 45 minutes, respectively
- it is expected that the inter-assay variation is greater than the intra-assay variation
- the intra-assay CVs aren't actually that bad (though we must bear in mind these are calculated from only two readings per sample), being < 10
- of greater concern are the inter-assay CVs, as 15 is a typical threshold
- intra and inter-assay variation decreases from 15, to 30, to 45 minutes, but as we have not evaluated the assay's precision to discriminate at these incubation times, this cannot be used to justify longer incubation times alone
- discrepancies at early time points propogate to late timepoints, and also to the second half of the plate, suggesting that a main source of variation is being introduced before the ELISA even starts
- the fact that the CVs of the unknowns correlate with the CVs of the positive and negative controls suggests a second source of variation that affects all wells of a plate uniformly. This could be coating, blocking, secondary concentration, or washing, but cannot be anything related to the sample